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ABSTRACT 



This edition of Research Brief examined the impact of five 
specific academic support programs at Indiana University- Purdue University 
Indianapolis on college student performance and persistence: ARCHE, a 
tracking and monitoring system for at-risk students; Student MAP was 
developed to assist students who place in the lowest level reading 
remediation class; Learning Communities, which links a first-year experience 
course with an entry-level discipline course; Horizons Workshops, which gives 
poorly performing students a second chance; and student mentoring, in which 
groups of students collaborate with a trained student mentor. The study found 
that: (1) students in ARCHE performed at levels similar to those not 

participating in the program; (2) the Student MAP program appears to have 
increased the performance of all students placed in the program; (3) the 
Learning Communities program, which has evolved considerably over several 
years, shows signs of having a positive impact on student performance and 
persistence; (4) the Horizons Workshops have had a dramatic effect on 
retention,, but the program does not appear to have affected grade performance 
in any significant way; and (5) the student mentoring program has had the 
most impact on students, raising student performance by as much as one-half a 
grade on average and increasing student retention by as much as 15 percent. 
(MDM) 
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Introduction 

lUPUI has made a strong commitment over its history to 
serve the Central Indiana community by making available 
the most comprehensive range of degree programs in the 
state to individuals of diverse demographic and academic 
backgrounds. The campus has maintained an '‘open- 
access'* admissions policy, while at the same time 
maintaining high standards for classroom performance 
and student progress. The balance between these two 
critical aspects of the mission — open access and high 
academic standards — may sometimes make the campus 
appear less than stellar in its performance according to 
such traditional measures as student retention. On the 
other hand, the fact that many students who were not 
eligible to enter any other public college or university in 
Indiana graduate alongside peers who started college with 
far greater academic and social advantage attests to the 
importance of providing these opportunities to the 
community. 

As part of its conunitment to diverse learners, lUPUI has 
developed an array of academic support programs to 
provide all students the best chance of succeeding. Many 
of these programs have been organized and administered 
through the Undergraduate Education Center (UEC). 

Now, as lUPUI prepares to evolve these program and 
services to their next level as the University College, it is 
important to consider the impact of these programs on the 
students who participate. This edition of Research Brief 
explores the impact of five specific academic support 
programs — ARCHE, Student MAP. Learning 
Communities, Horizons, and Student Mentoring — on 
student performance and persistence. Future studies will 
include Academic Aerobics, and Square One along with 
other initiatives. 

Method 

Each of the five academic support programs is analyzed 

aV'pUI J UUk A\ytt\yVT,9 U WWIAUtlWlA AWAUiUk. X kkX>» AWAAAAUA 

begins with a description of the program, including its 
program content and target audience. This is followed by 
a description of the method used to identify one or more 
comparison groups of “untreated** students, and a 
statistical profile of both the target and comparison groups 
in terms of background, enrollment, performance and 
persistence indicators. 



HIGHLIGHTS 



This edition of Research examines the impact of five 
specific academic support programs on student performance 
and persistence. These five programs — ARCHE, Student 
MAP, Learning Communities, Horizons, and Student 
Mentoring— are among those being incorporated into 
University College* 

The ARCHE program, a tracking system for monitoring 
many of lUPUI’s “at-risk** students^ has demonstrated that 
although TUPUI has admitted l^ger numbers of under- 
prepared students over the last few yeais, these student are 
performing at similar levels to tlheir. earlier counterparts. 

The Student MAP program was developed to assist a 
subset of students who place into the lowest level reading 
remediation class (XI50). Since its inception, it appears 
that all students placed into XISO have performed 
increasingly better on average. , 

The Learning Communities program has evolved 
consider^ly over the:past two yeami-most notab by its 
recent linkage to the first-year experience course. These 
changes m^e it difficult to assess the progii^*^^^^ on 
student^^; The most recent yersioni^ii^ Fall 

1 996^ stip^^ si^ of having a^ in^)abt on i student 
perform^ce and persUtence, but itiis too early 
firm conclusions. , . 

Horizdiis Workshops provide inihimal supp 
perfbniuhg smdente^ tojgiye tlwm a second chi^ce^t^^^^ ‘ 
improve their academic iperfoimance.^^ T^ a 

draniaidiceffect on retention, partly because^^ 
provid^ w as opposedito beingfdismissed, but 

it doesiinot a|^ toafftet grade performanc any 
significant way. ■ 

The Student Mehtbring Pr^ largest of the 

impressive effect on stiident performance and persistence. It 
is responsible for raising student perfonnai^jby a^ as 
one-half a grade (0.50 GP A) on average^a^ inerting 
studentj^ter^bnl^ 

smdentsYsatisf^ lUPUI 

primary :thr^ snidyihg with oAct students: ou^ide class. 
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Since it is infeasible to derive completely matched 
samples, a second phase of analysis statistically controls 
for remaining differences between target and comparison 
groups that may otherwise explain differences in 
performance and persistence. Linear and logistic 
regression with block entry of predictors, and analysis of 
covariance were the primary statistical techniques used to 
factor out remaining differences. In addition, 
comparisons are made between specific matched 
subgroups among target and comparison samples. Finally, 
for the larger programs included in this analysis, 
differences in student satisfaction are examined using 
student responses from the Spring 1996 Continuing 
Student Satisfaction and Priorities Survey. 

The background, enrollment, performance, and 
persistence indicators employed in this study vary among 
the programs due to timing issues, differences in group 
composition, and availability of data. Appendix 1 
summarizes these indicators, including the statistical test 
used for assessing baseline group differences, and the 
programs to which each indicator was applied. 



Results 

The ARCHE Program 

The ARCHE program was established in Spring 1994 as a 
tracking system for monitoring the academic progress of 
“at-risk’* students in the UEC preparatory program. The 
program is not an intervention, per se, but rather a 
monitoring function to establish a baseline for assessing 
the effects of intervention programs such as the Student 
MAP and Learning Communities programs included in 
this evaluation study. The program was created to ensure 
accurate data were available for comparing to other 
student populations. The ARCHE tracking program , 
isolates a research sample composed of students who 
place into a remedial reading course (XISO or X1S2) from 
among all freshmen UEC preparatory program new admits 
who have no transfer hours. 

Baseline Group Comparisons 

This analysis begins with the ARCHE program to examine 
more closely the baseline population for subsequent 
interventions. In addition, a comparison group was 
identified as students fitting the ARCHE profile who 



Table 1. Baseline Comparisons for the ARCHE Program 
Backgrbuhd and ErirollinehtiCharacieri^t^ 





Fall 1995 Participants 


Fall 1996 Participants 


Fall 93 Comp. Group 




Indicator 


n’ 


sigf 


Indicator 


n' 


sigf 


Indicator 


n' 


Avg. Age 


19.4 


378 


*** 


19.5 


527 


*** 


20.4 


511 . 


Percent Female 


54% 


378 


* 


61% 


527 




61% 


511 


Percent Afric. Amer. 


20% 


378 


*** 


26% 


527 


*** 


12% 


511 


Avg. HS % Class Rank 


36.6 


334 




38.9 


459 




37.2 


453 


Placed into Rem Math 


91% 


378 




92% 


527 




89% 


509 


Placed into Rem Writing 


75% 


377 


*** 


55% 


512 




53% 


505 


Placed into Rem Reading 


100% 


378 


4 


100% 


527 


4 


100% 


511 


Avg. Semester Hours 


10.6 


378 


*** 


10.5 


527 


*** 


9.7 


511 


Performance and':Persistehce:*<Unadjusted 


Sem GPA 


1.82 


344 




1.95 


509 




1.91 


458 


ENG W001 


2.12 


146 




2.12 


209 




2.35 


187 


ENG W131 


2.11 


56 




2.18 


123 




2.05 


129 


MATH M010 


1.60 


87 




1.53 


49 




1.56 


106 


MATH 001 


1.97 


168 




1.78 


304 




1.77 


180 


MATH 1 1 1 


1.33 


24 


*** 


2.05 


38 


* 


2.66 


36 


PSYB104 


2.44 


44 


* 


1.64 


34 




1.79 


42 


Retn to next sem 


72% 


378 




76% 


527 




71% 


511 


Retn to next year 


52% 


378 




na^* 


- 




50% 


511 



Valid n (exciuding missing values) upon which indicator is based. 

^See Appendix 1 for details on the statistical tests used to assess group differences. 

^e difference relates not to the percent requiring remediation overall, but to the percent placed in 001 vs. M010: 
39% of the comparison group placed into M010 compared to only T^p of the participant group 
^Placement in a remedial reading course was an eligibility requirement for the program and comparison group. 
®The one year retention rate for the Fall 1996 cohort cannot be determined until Fall 1997. 

•p<.05; ••p<.01; ***p<.001 
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Table 2. Effect of Program Participation on Semester GPA when 
Controlling for Other Predictors of GPA 





Unstandardized Coefficients 
Beta Std. Error 


t-value 


Sig. Level 


(Constant) 


0.401 


0.349 


1.15 


0.251 


Sex 


0.171 


0.079 


2.15 


0.032 


AGE 


0.052 


0.012 


4.23 


0.000 


Semester Hrs 


0.031 


0.014 


2.23 


0.026 


HS % Rank 


0.013 


0.002 


7.07 


0.000 


Ethnic Status' 


-0.464 


0.101 


-4.58 


0.000 


Math Placement^ 


-0.393 


0.129 


-3.05 


0.002 


PROGRAM'’ 


0.161 


0.078 


2.07 


0.038 


Without Program variable, R = .325, = .105, F(5,841) = 19.824; P < .001 

With Program variable, R = .332, R* = .1 10, F(6,840) = 17.285; P < .001 



’Dummy variable with values, 0 = not African American; 1 = African American 
^Dummy variable with values, 0 = college level; 1 = remedial (M010 or 001) 
^Dummy variable with values, 0 = comparison group; 1 = ARCHE program 



began in Fall 1993, before the program was 
established. Although ARCHE students 
receive no direct intervention, many of the 
students tracked through this program 
participated in such interventions as Student 
MAP and Learning Communities. 

Therefore the comparison establishes a 
baseline for later subgroup comparisons 
among participants in these other programs. 

Table 1 displays the baseline background, 
enrollment, performance and persistance 
indicators for the Fall 1995 and Fall 1996 
ARCHE cohorts in comparison with Fall 
1993 students who met the same eligibility 
criteria. Despite using the same criteria, 
there are several notable differences 
between the ARCHE and comparison 
groups. Specifically, both ARCHE cohorts 
contain higher percentages of African 
American students, nearly double the rate among the Fall 
1993 comparison group. Furthermore, ARCHE students 
are younger and have higher average credit hours in their 
first semester. These two trends follow an overall 
changing pattern in the lUPUI undergraduate student body 
over the past three years. 

Unadjusted Outcome Differences 

There are few significant differences in the performance 
and persistence measures between the ARCHE and 
comparison groups. For the ARCHE group, performance 
in MATH 1 1 1 was significantly lower among the few 
students who enrolled in that level of math, compared to 
the similarly small group of 1 1 1 enrollees from the Fall 
1993 comparison group. On the other hand, the Fall 1995 
group outperformed their 1993 peers in the introductory 
Psychology course, B 104. The lack of differences in this 
comparison suggests that despite some changes in the 
demographic characteristics of this at-risk group, 
performance and persistence rates have not changed much 
over the past four years. 

Further Analyses 

The presence of demographic changes may in fact be 
suppressing differences in performance and persistence. 

To iesi this hypothesis, regression analyses were 
conducted on both semester GPA (linear), and retention to 
the next semester (logistic). For the purpose of this 
analysis, the Fall 1995 and Fall 1996 ARCHE groups 
were combined to compare with the Fall 1993 group. The 
strongest predictors from among the background and 
enrollment characteristics variables were first entered into 
the model followed by the dummy variable indicating 
whether the student was in the ARCHE program or 
participant group. 



Table 2 summarizes that results of the regression on 
Semester GPA, showing that the significant predictors 
together accounted for 10.5% of the variance in GPA. The 
program variable added significantly to the prediction of 
GPA, although contributing only a modest 0.5% to the 
total variance predicted. TTie unsUindardized regression 
coefficient and standard error, suggests that membership 
in the program group added on average 0.16 grade- 
points — the 95% confidence interval for this program 
effect is 0.01 to 0,31 grade-points. 

A similar result is obtained by treating the above 
predictors as covariates in a simple factorial general linear 
model analysis of the differences in Semester GPA by 
group membership. After controlling for the covariates, 
there exists a significant main effect for group 
membership (F(l,840) = 5.168, p < .05), with adjusted 
mean GPAs of 1.91 for the comparison group and 2.01 for 
the ARCHE group. 

A logistic regression was conducted to assess whether the 
same hypothesis held true with regard to retention rates, 
that is, whether the demographic differences were 
suppressing the impact of group participation on retention. 
Tlie analysis did not yield evidence to support this 
hypothesis — group membership did not add to the 
prediction of retention or withdrawal after controlling for 
the strongest predictors (GPA, Credit Hours, H.S. rank, 
and ethic status). 

Conclusions 

Although the ARCHE program does not represent an 
academic support intervention, there is a small but 
significant difference between ARCHE program 
participants and their peers from prior years in overall 
grade performance. There was no difference in persistence 
rates between groups. In effect, one could say that despite 
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changes in certain demographics that would normally be 
associated with declines in performance, students 
monitored through the ARCHE program fared as well as 
their earlier peers. Part of the reason for this may be that 
many of these students participated in the interventions 
examined in the next few sections of this report. 

Student Mentoring/Advising Program (MAP) 

The Undergraduate Education Center, the Office of 
Scholarships and Financial Aid, and the Student Mentor 
Program joined in an effort to develop a pilot program for 
Fall 1996 to serve students who score at or below the 10* 
grade reading level on the lUPUI placement test when 
they enter the institution. Six sections of Education XI 50, 
lUPUI’s lowest reading level course, were selected for the 
pilot. Students who are enrolled in those sections are 
offered an opportunity to meet with a student mentor 
outside of class for IVi hours each week. The student 
mentor (student MAP) introduces participants to campus 
support systems, provides mentoring support, takes 
students on tours of facilities on the campus, and serves as 
a guide and friend to students in the program throughout 



the semester. Two UEC counselors work directly with the 
program. Two incentives are offered to students to 
encourage participation. First, the Financial Aid office 
creates loan*ffee packages for all students who participate 
in the program if they meet established criteria. One 
criterion is that the students agree to enroll in 9 or fewer 
hours during the semester that they participate in the 
program. Secondly, all students who meet regularly with 
their Student MAP are eligible for scholarship book 
awards at the end of the semester, based on their GPA. 

Baseline Group Comparisons 

Two groups were identified for comparison purposes. 

The first group is composed of other UEC-P students 
enrolled in XI 50 during the Fall 1996 semester. In some 
ways, this represents students who could have but chose 
not to participate in the program. The second comparison 
group is composed of M Fall 1995 UEC-P students who 
placed into the X150 course. 

Students in the MAP program appear similar to Fall 1996 
non-participants with two exceptions. Following the 



Table 3. Baseline Comparisons for the MAP Program 



Babkgrbund arid Enr6iimeht:Characteiistlcs^;a 





Participants 


sig.‘ 


Non-Participants 


sig.^ 


Comparison Group 


Indicator 


n’ 


indicator 


n’ 


Indicator 


n’ 


Avg. Age 


18.9 


88 




19.6 


125 




19.9 


158 


Percent Female 


61% 


88 




61% 


125 




53% 


158 


Percent Afric. Amer. 


42% 


88 




29% 


125 




31% 


158 


Avg. HS % Class Rank 


36.8 


88 




30.2 


125 




34.2 


158 


Apply for Fin Aid 


76% 


88 


** 


58% 


125 




69% 


158 


Avg Pet Need Met 


68.7 


67 




42.1 


73 


* 


41.1 


108 


Placed into Rem Math 


88% 


88 




91% 


124 


e**3 


93% 


155 


Placed into Rem Writing 


74% 


87 




58% 


121 


*e 


89% 


157 


Placed into Rem Reading 


100% 


88 




100% 


122 




98% 


155 


Avg. Semester Hours 


11.4 


88 


** 


10.3 


125 


e 


10.6 


158 


Pet in ARCHE 


92% 


88 




90% 


125 




86% 


158 


Performance and Persistence ' Uriadjusted ^^^ 




Avg GPA 


2.05 


81 




1.89 


122 


* 


1.69 


140 


ENG W001 


2.35 


51 




1.96 


51 




2.13 


65 


EDUCX150 


2.08 


70 




2.30 


110 




4 




EDUCX151 


2.30 


41 




1.93 


53 




4 




MATH 001 


1.79 


43 




1.60 


75 




1.78 


63 


Retn to next sem 


76% 


88 




78% 


125 




68% 


158 



Valid n (excluding missing values) upon which indicator is based. 

^Statistitical significance relates to the difference between the participant and non-participant groups. 

^See Appendix 1 for details on the statistical tests used to assess group differences. 

^he difference between groups relates more to the percent of students placing into M010: 34% of comparison group 
compared to 11% of participants and 6% of non-participants. 

^Only grades of Satisfactory and Unsatisfactory were conferred through the Fall 1995 semester 
•p<.05; ••p<.01 ; •**p<.001 
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relationship of this program with the financial aid process, 
MAP students are far more likely to have applied to and 
received financial aid. They also enroll in slightly more 
credit hours on average. The financial aid difference 
disappears when compared to the more inclusive Fall 

1995 comparison group. However, the MAP group has a 
larger proportion of their financial need met (as part of the 
program) and also appears to average slightly higher 
levels of placement into non-remedial math (including 
111) and writing courses (although the vast majority of all 
these students are placed into remedial sections in all three 
placement areas). Table 3 summarizes these comparisons, 
also showing that the vast majority of all groups 
considered are part of the ARCHE tracking system. And, 
although not statistically significant, the MAP program 
includes a notably higher proportion of African American 
students 

Unadjusted Outcome Differences 

Table 3 also shows one notable difference in outcomes 
between MAP students and the Fall 1995 comparison 
group — a significantly higher semester GPA. Although 
this GPA is higher for the MAP group compared to Fall 

1996 non-participants, this difference is not statistically 
significant. It is also interesting to note that the difference 
in GPA between the entire Fall 1996 group (MAP 
participants and non-participants, combined) is marginally 
significantly higher than for the Fall 1995 group (F(l,341) 
= 3.823, p=.051). 

Further Analyses 

To further explore the impact of the MAP program on 
student performance, the same statistical techniques of 
linear regression and analysis of covariance for semester 
GPA, and logistic regression for retention, were 
performed. These analyses were conducted with the Fall 
1996 groups, participants and non-participants. High 
school percentile rank was the only significant predictor 
of semester GPA, and course load and semester GPA were 
significantly associated with retention. However, program 
participation did not add to the prediction of either 
performance or persistence. 

Finally, grades and retention rates were compared among 
the African American program participants and non- 
pariicipariis <uiiung the Fail 1996 cohort. Tne average 
grades were higher among program participants (1.77 vs. 
1.59) and the retention rate lower (70% vs. 78%), but 
neither of these differences was statistically significant. 

Conclusions 

With the MAP program established, students placing into 
Education X150 appear to do better with regard to their 
overall semester grades compared to their counterparts 
from the previous year. The very early view we have of 



retention among these groups (i.e., one semester), shows 
no program impact. 

Learning Communities 

First-year experience courses have been in place for a 
number of years, but in Fall 1995 the School of Liberal 
Arts and the Undergraduate Education Center altered the 
existing program significantly. A model was developed 
that linked a first-year-experience course with an entry- 
level discipline course. The new model was presented to 
students as a Learning Community, with both sections 
taught by the same faculty member. In addition, an 
instructional team was added to provide even greater 
support to entering students. Each instructional team was 
composed of five members including the faculty member, 
an academic advisor, a librarian, a student mentor, and a 
technical support staff member. The model expanded in 
Fall 1996 with the Schools of Science and Engineering 
and Technology joining the program by offering their own 
specific model of the first-year-experience course. 

Baseline Group Comparisons 

Students participating in the Learning Communities 
program were compared with other students enrolled in 
the courses to which first-year experience courses were 
“linked,** but were not enrolled in the first-year experience 
course. Table 4 shows the comparison of Learning 
Community students with their comparative peers for each 
of the last three semesters. Fall 1995, Spring 1996, and 
Fall 1996. The differences in background and enrollment 
characteristics were generally small and differed from one 
semester to the next. In Fall 1995, participants included a 
higher percentage of African Americans, and slightly 
older students taking slightly fewer credit hours on 
average. The Spring 1996 groups differed only according 
to age, with the participants averaging nearly two years 
older than non-participants. For the Fall 1996 group, the 
participants included higher proportions of students with 
college level reading abilities and who placed into college 
level writing. 

Unadjusted Outcome Differences 

If the differences between background and enrollment 
characteristics are considered few, the differences in 
pCi iCrTTiuriCC and persistence outcornes are even fewer. 

Fall 1995 participants averaged significantly lower grades 
and a significantly lower rate of retention to the next 
semester (although not to the next year). There were no 
significant differences in performance and persistence for 
the Spring 1996 groups. Fall 1996 participants generally 
averaged higher grades than non-participants, but these 
differences were not statistically significant. Fall 1996 
participants did return for the Spring 1997 semester at 
significantly higher rates than did non-participants. 
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Table 4. Baseline Comparisons for the Learning Communities Program 
Background and Enrollment Characteristics . 





F 


all 19< 


)5 


Sorina 1996 


Fall 1996 


Participants 


sig.^ 


Non-Partic. 


Partcipants 




Non-Partic. 


Participants 


sig.^ 


Non-Partic. 


Indicator 


n' 


lr)dicator 


n' 


Mcator 


n' 


Mcator 


n’ 


Indicator n ' 


Indicator 


n' 


Avg. Age 


21.5 


133 


* 


20.5 


924 


23.5 


95 


•• 


21.6 


262 


19.9 


309 




20.0 


1193 


Percent Female 


53% 


133 




54% 


924 


45% 


95 




49% 


262 


56% 


309 




55% 


1193 


Percent Afric. Amer. 


23% 


133 


•* 


13% 


924 


14% 


95 




18% 


262 


14% 


309 




17% 


1193 


Avg. HS % Class Rank 


35.3 


103 




38.6 


813 


35.1 


75 




35.6 


204 


38.1 


270 




38.7 


1033 


Apply for Fin Aid 


60% 


133 




63% 


924 


54% 


95 




50% 


262 


65% 


309 




61% 


1193 


Avg Pet Need Met 


33.6 


eo 




36.5 


580 


39.4 


51 




36.0 


131 


40.7 


200 




41.3 


723 


Placed into Rem Math 


90% 


127 




90% 


912 


94% 


95 




92% 


261 


91% 


305 




89% 


1182 


Placed into Rem Writing 


61% 


130 




59% 


900 


39% 


95 




49% 


260 


30% 


305 


• • 


40% 


1163- 


Placed into Rem Reading 


31% 


128 




35% 


904 


20% 


95 




27% 


260 


26% 


305 


• •• 


38% 


1172 


Avg. Semester Hours 


9.7 


133 


••• 


10.8 


924 


10.0 


95 




9.9 


262 


11.7 


309 


• •• 


10.7 


1193 


Perrofmance and Persistence - Unadjusted : : . , 






Avg GPA 


1.77 


131 


** 


2.09 


866 


2.05 


95 




2.00 


245 


2.03 


301 




1.94 


1137 


ENG wool 


2.01 


26 




2.35 


336 


2.27 


28 




2.07 


65 


2.27 


58 




2.17 


331 


ENG W131 


2.14 


17 




2.23 


237 


1.90 


51 




1.92 


79 


2.23 


138 




2.10 


34h- 


EDUCX150 


4 






4 




1.52 


11 




1.91 


15 


2.36 


15 




2.23 


131 


EDUCX151 


4 






4 




1.90 


3 




2.18 


153 


2.41 


59 




2.07 


651. 


EDUCX152 


4 






4 




2.77 


6 




2.67 


26 


2.62 


59 




2.44 


157 


MATH M0 10 


1.68 


32 




2.17 


203 


2.67 


15 




1.73 


52 


1.98 


24 




2.02 


89 


MATH 001 


1.66 


42 


• 


2.11 


426 


1.98 


51 




1.92 


120 


2.01 


183 




1.84 


681 


PSY B104 


1.59 


27 


* 


2.27 


171 


1.51 


9 




1.77 


38 


1.92 


87 




1.63 


200 


Retn to next sem 


68% 


133 


* 


77% 


924 


61% 


95 




56% 


262 


80% 


309 


• 


74% 


1193 


Retn to next year 


48% 


133 




54% 


924 


49% 


95 




42% 


262 


na^ 


- 




na^ 


- 



valid n (excluding missing values) upon which indicator is based. 

^See Appendix 1 for details on the statistical tests used to assess group differences. 

*The one year retention rate for the Fall 1996 cohort cannot be determined until Fall 1997. 
*Only grades of Satisfactory and Unsatisfactory were conferred through the Fall 1995 semester 
•p<.05; ••p<.01; •••p<.001 



Further Analyses 

Since the Learning Communities program has changed 
significantly over the past three semesters^ additional 
analyses were conducted focusing only the Fall 1996 
cohort. Linear regression analysis established age, H.S. 
rank, and ethnic status as the significant predictors of 
semester GPA, accounting for just under 10% of the 
variation, combined. Participation status did not 
significantly contribute to predicting grades after 
controlling for these factors. With regard to predicting 
semester retention, the significant difference between 
Learning Community participants and non-participants 
disappears when controlling for the other predictors of 
retention, specifically, semester GPA, credit hours taken, 
percent of financial need met, and ethnic status^ 

Given the large and diverse composition of students 
participating in Learning Communities, final analyses 
focused on the impact of participation among two specific 
subgroups of students: student who placed in remedial 
reading and African American students. Among the 



’ Assessed using logistic regression on enrollment status in the 
following semester with block entry first of the predictor variables and 
then program participation status. 



students who placed into remedial reading. Learning 
Community participants averaged a slightly higher 
semester GPA (2.13 vs. 1.91), but this difference was not 
significant (F(l,497) = 2.055, p = .152). Similarly, the 
participant group had a higher semester retention rate 
(85% vs. 77%), but again, this difference was not 
statistically significant (chi-square(l) = 2.173, p = .140). 
The same pattern held among African American students. 
Learning Community participants averaged higher grades 
(1.96 vs. 1.58, F(l,236) = 3.841, p = .051), and retention 
rates (84% vs. 78%, chi-square(l) = .751, p = .386), but 
neither difference was statistically significant. 

More recent data on persistence shows that among the Fall 
1995 Learning Communities cohort, 37 percent were 
enrolled for the Spring 1997 semester. Perhaps more 
significantly, 41 percent of the African American 
participants, and 53% of the male African American 
participants were enrolled for the spring 1997 semester. 
These data suggest that the program may have a 
disproportional positive impact for African American 
males, the group with the lowest levels of persistence at 
lUPUl. While the numbers are still very small, this 
retention rate is significantly higher compared not only to 
other African American males but to all other lUPUl 
undergraduates. 
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Conclusions 

The Learning Communities program has changed enough 
over the past three semesters that comparisons are 
difficult. There are some signs that the Fall 1996 program 
has been the most successful: grades and persistence are 
slightly higher although the impact of persistence is 
attenuated when controlling for other factors. 

While complete data are not available to conduct analyses, 
a review of registration dates suggests that the students in 
learning communities are among those who were admitted 
and registered relatively late in the registration cycle. 

This is important only because other reports have claimed 
lowered levels of academic achievement and persistence 
for “late-deciders” and others who come late to the 
registration process. Subsequent analyses will focus 
exclusively on students who have enrolled in Joining the 
Scholarly Community sections (COAS 200, Windows on 
Science, Tech 105, and other courses). Students in these 
sections have lower withdrawal rates than do comparable 
groups of students, and we need to determine whether 
more long-term effects accompany this increased 
persistence. Both quantitative and qualitative research is 
warranted. 

Horizons Workshop 

The Horizons Workshop was developed for students who 
have a cumulative GPA of 1.24 or below and less than 18 
total GPA credits in their first semesters of enrollment. 

The sessions were designed to help students understand 
the reasons for attending the workshop, 
for their probationary status and for 
attainment of low GPAs. In addition, 
session leaders address topics 
including how GPAs are computed and 
what is necessary to meet UEC 
Retention Standards. Other topics are 
time management techniques; 
availability of university resources 
such as mentoring, tutoring, and 
counseling; and test-taking and study 
skills techniques with handouts 
supporting each topic. 

Baseline Group Comparisons 

This analysis evaluates the 
performance in the spring 1996 
semester of students who were enrolled 
in the Horizons Workshops based on 
their Fall 1995 academic performance. 

The comparison group is composed of 
students who met the same eligibility 
requirements based on their Fall 1993 
academic performance who then 
enrolled for the Spring 1994 semester. 



It should be noted that changes in the academic dismissal 
policy impact the comparability of the two groups. A 
portion of the Fall 1993 students (those with 9 or more 
total GPA hours attempted) who performed so poorly 
would have been dismissed prior to the spring 1994 
semester. 

Table 5 summarizes the baseline differences between the 
Horizon and comparison groups. As noted in the ARCHE 
program analysis, the two biggest differences — Horizons 
students slightly younger on average and enrolled in more 
credit hours — follows the general trend for lUPUI 
undergraduates over the past four years. In addition to 
these differences, the Horizons group has a slightly lower 
rate of placement into remedial mathematics courses, 84% 
compared to 91% for the comparison group. 

Unadjusted Outcome Differences 

The Horizons and Comparison group had very similar and 
extremely low average Fall GPAs. And, although the 
Horizons group averaged a slightly lower spring semester 
GPA, this difference is not statistically significant. It 
should also be noted that a number of students in the Fall 
1993 comparison group, presumably the poorest academic 
performers, were dismissed for academic reasons and not 
given a chance to go through such prograihs as Horizons 
in an attempt to improve their grades. 

In perhaps the most striking program effect evident in this 
study. Horizons students are retained over a one year 
period at over twice the rate of the comparison group. As 



Table 5. Baseline Comparisons for the Horizons Program 



:Baiikigroiilrid;arid:Enro 





1995-96 Participants 


sig.^ 


1993-94 Comp. Group 


indicator 


n' 


indicator 


n’ 


Avg. Age 


19.5 


150 


mm 


21.4 


119 


Percent Female 


48% 


150 




54% 


119 


Percent Afric. Amer. 


23% 


150 




23% 


119 


Avg. HS % Class Rank 


39.5 


127 




34.4 


105 


Pet in Prep Program 


77% 


150 




84% 


119 


Pet Beginner 


90% 


150 




86% 


119 


Placed into Rem Math 


84% 


146 


* 


91% 


117 


Placed into Rem Writing 


53% 


143 




39% 


9 9 


Placed into Rem Reading 


41% 


143 




50% 


113 


Avg. Semester Hours 


11.1 


150 


*** 


9.8 


119 


Avg. Fall GPA 


0.61 


150 




0.63 


119 


Performance and Persistence • Unadjusted ; 


Spring GPA 


0.81 


150 




1.02 


97 


Retn to next sem 


41% 


150 




32% 


119 


Retn to next year 


35% 


150 


*** 


16% 


119 



Valid n (excluding missing values) upon which indicator is based. 

^See Appendix 1 for details on the statistical tests used to assess group differences. 
•p<.05; ••p<.01; •••p<.001 
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noted above, this is partly because 

of the change in academic dismissal Table 6. Sub-group Differences in Grades and Retention among Horizon and 
policy. Comparative students 



SubarouD 


Indicator 


Horizons 


Comp. Grp 


Sig. 


UEC-P 


Percent of Students in Group 


77% 


847o 






Average Spring GPA 


0.83 


0.86 






One-year Retention Rate 


357o 


167o 




Remedial 


Percent of Students in Group 


41 7o 


50% 




Reading 


Average Spring GPA 


0.76 


0.67 






One-year Retention Rate 


367o 


207o 


t ■ 


African 


Percent of Students in Group 


237o 


237o 




Americans 


Average Spring GPA 


0.71 


0.87 






One-year Retention Rate 


357o 


157o 


t 



tp<.10; ""p<.01 

Note, Chi-square test for independence used for the percent of group and 
retention rate analyses of group differences; one-way ANOVA (F-test) used to 
test for differences in grades. 



Further Analyses 

The only measure that predicted 
spring semester grades was fall 
semester grades, and this 
relationship was fairly modest 
(Pearson correlation of 0.23, 
p<.01)^ As would be expected 
given the similarity of fall grades 
among the two groups, controlling 
for fall grades does not effect the 
impact of program participation on 
spring grades. 

As shown in the baseline 
comparison, Horizons participants 
had a significantly and substantially 
higher retention rate than among 
the comparison group. Spring GPA 
was the only other significant 
predictor of retention with higher 
grades being associated with a greater likelihood of 
returning. Since the Horizons group averaged lower 
average grades than the comparison group (although not 
significantly so), controlling for spring grades only 
increases the effect size of program participation on 
retention, although by only a very small amount (by less 
than one-quarter of one percentage point). 

A final analysis looked at the differences in spring grades 
and one-year retention rates among three subgroups 
among Horizons and the comparison group: Students in 
the UEC preparatory program, students who placed into 
remedial reading, and African American students. 

As Table 6 shows, the lack of differences in grades and 
large difference in retention rates holds up among all 
subgroups. The significance level for the retention rate 
difference does become weaker as a result of the 
decreasing group sizes. It is also interesting to note that 
the lower average grades for the Horizons group apparent 
in the overall summary table changes for two of the 
subgroups. The gap narrows to almost identical averages 
among the UEC-P subgroups, and the direction of the 
non-significant effect reverses for students who placed 
into remedial reading. 



The ability to statistically predict spring grades was dampened by the 
lack of variation in this measure. Two out of every five students in the 
Horizons and comparison groups had a 0.00 GPA. 



Conclusions 

The Horizons workshops appear to have a very strong 
impact on the retention of these poorly performing 
students. However, it does not appear to affect their 
academic performance, at least in the short time span 
considered in this study. It will be important to continue 
tracking these students to see if they can bring their grades 
up far enough to be able to continue their studies. 

Peer Mentoring 

The Student Mentor Program operates with groups of 
students working together with a student mentor to belter 
understand difficult course material. This collaboration 
provides an enriching environment where students 
strengthen academic skills and build community. Student 
mentors are trained to guide the students through the 
learning process, not to provide traditional tutoring. The 
Student Mentor Program provides mentoring in a variety 
of courses that have been traditionally difficult for 
students and often have a high failure rate. Mentoring is 
provided in over 50 courses, and there are 140 students 
serving as mentors in the 1996-97academic year. 

Baseline Group Comparisons 

Students were classified as taking part in the Student 
Mentoring Program if they attended at least three 
mentoring sessions in association with a single course. 

The comparison group includes all students enrolled in 
mentoring courses that did not take advantage of the 
mentoring sessions at least three limes. It should also be 
noted that students could be enrolled in more than one 
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mentonng course. As long as they attended three sessions 
for one of these courses, they were considered to have 
taxen part in the program 

Table 7 summarizes the differences between program 
participants and non-participants according to the various 
background, enrollment, performance, and persistence 
indicators available for this group. Since the student 
ntentonng program is open to all students enrolled in 
these very popular courses, the pool of eligible 
pmicipants is both large and diverse, covering students in 
all schools, at all class levels, and of varying ability levels. 

As Table 7 shows, the self-selected participants differ 
^om non-participant peers along several critical 
dimensions: on average, they are older, have higher 
s^ng GPAs, are less likely to be freshmen, and less 
ikely to be in the UEC-preparatory program. In other 
words, many relatively better prepared students take 
advantage of the program. There are also differences for 
o semesters in the participant group being more likely 
to have applied for financial aid and to have had a larger 
proportion of their financial need met. Finally, for the Fall 
1^95 cohort, participants included proportionately more 



females and fewer minorities, but neither of these 
ifferences held for the spring semester. 

Unadjusted Outcome Differences 

program participants 

hnarrf H%f surprising to see across the 

board differences in the performance and persistence 

indicators. Participants in the Student Mentoring Program 
ad significantly higher semester grades overall, as well as 
in their mentoring courses. They were also more likely to 
complete their mentonng courses, to re-enroll in the next 
semester, and a year later. 

F urther Analyses 

Following the strategy used in earlier analysis, linear 
regression analysis was used to determine which 
background and enrollment indicators contributed to the 
prediction of semester GPA so that these factors could be 
controlled in evaluating program impact. As one might 
expect, the single best predictor of semester GPA was 
pnor GPA, which alone accounted for just under one- 
quarter of the variation in semester GPA. Adding high 




^valid n (excluding m.ss.ng values) upon which indicator is based ' 

‘Placements evaluated for beginning freshmen 
•p<.05; ••p<.01; •••p<.ooi 
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school rank, English placement, math 
placement, minority status (minority vs. 
non-minority), and class level (freshman 
vs. all other), brings to 27% the total 
variation accounted for in semester GPA. 

After controlling for these factors, 
program participation still contributes 
significantly to the predication of semester 
GPA, adding another 0.5% to the 
prediction. Entering the predictor 
variables as covariates in a simple factorial 
analysis of variance shows that the group 
difference in semester GPA changes to 
2.58 for participants vs. 2.26 for non- 
participants for the fall and spring 
semester groups, combined. The program 
affect on GPA, when controlling for 
differences in group profile, is 
approximately one-third of a letter grade, 
or 0.33 grade-points. 

Unfortunately, relying on prior GPA as a 
predictor restricts the analysis to students 
who have some prior college experience. 

To circumvent this limitation, subgroup 
analysis was used, focusing specifically on 
first-time college students (beginner 
freshmen) and, within that group, students who placed into 
remedial reading, and students in the UEC preparatory 
program. For these analyses, the Fall 1995 and Spring 
1 996 groups were merged to provide more power through 
larger sample sizes. Tables 8 through 10 summarize these 
analyses. 



•••p<.001 

Table 8 shows that when looking at only first-time 
freshmen, many of the demographic and enrollment 
differences seen in the larger group are diminished. When 
controlling for the remaining differences, the difference in 
spring GPA reduces from 0.59 grade points to 0.50 grade 
points, still a highly significant difference. The 16 
percentage point difference in one year retention rate is 
reduced by only 2 to a 14 percent gap, 
which is also still highly significant. 

This same pattern holds true when looking 
at first-time students in the UEC 
preparatory program (Table 9), and when 
looking only at first-time students who 
placed into remedial reading (Table 10). 

In both cases the differences between 
participants and non-participants in 
background and enrollment characteristics 
are very small and mostly unrelated to 
factors which predict performance and 
persistence. At the same time, the 
differences in outcomes are as large, or 
larger than for the overall group. This is 
especially noticeable among students who 
place into remedial reading, where 
program participants achieve semester 
GPAs nearly a grade higher on average 
than students who do not participate in the 
program. 



Table 9. Peer Mentoring Comparisons Among First-Time 
UEC Preparatory College Students 
jacl^rourid and Enrollment Characteristics 





Participants 


sig. ^ 


Non-Partic. 


indicator n ^ 


indicator 


n’ 


Average Age 


23.6 


126 


... 


20.7 


1460 


Percent Female 


53% 


126 




50% 


1460 


Percent Minority 


19% 


126 




19% 


1460 


Avg. HS % Class Rank 


41.3 


102 


* 


36.4 


1229 


Avg Pet Need Met 


41.2 


90 




35.3 


837 


Placed into Rem Math 


95% 


126 




90% 


1445 


Placed into Rem Writing 


54% 


123 




58% 


1434 


Placed into Rem Reading 


35% 


124 




33% 


1438 


Avg. Semester Hours 


10.8 


126 


* 


10.2 


1460 


Performance and Persistence - Unadjusted 




Avg Semester GPA 


2.63 


125 


*** 


1.92 


1391 


Avg. Mentor Course GPA 


2.67 


122 


*** 


1.91 


1311 


Retn to next year 


64% 


126 


*** 


38% 


2302 



Valia n (excluding missing values) upon which indicator is based. 

See Appendix 1 for details on the statistical tests used to assess group differences. 
•p<.05: **p<.01; •••p<.001 
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Table 8. Peer Mentoring Comparisons Among First-Time 
College Students 

Background and Enrollment Characteristics 





Participants 




Non-Partic. 


Indicator 


n> 


sig.^ 


Indicator n ^ 


Average Age 


22.0 


194 


... 


22.1 


2302 


Percent Female 


57% 


194 




53% 


2302 


Percent Minority^ 


14% 


194 




16% 


2302 


Avg. HS % Class Rank 


53.5 


164 


* 


48.8 


2008 


Avg Pet Need Met 


35.9 


137 


*** 


48.7 


1355 


Placed into Rem Math 


74% 


191 




76% 


2267 


Placed into Rem Writing 


44% 


189 




48% 


2234 


Placed into Rem Reading 


25% 


187 




27% 


2238r^ 


Pet in Prep Program 


65% 


194 




63% 


2302^ 


Avg. Semester Hours 


11.7 


194 


*** 


10.8 


2302 


Performance and Persistence TiUnadJustec 






Avg Semester GPA 


2.76 


193 


*** 


2.17 


2209 V 


Avg. Mentor Course GPA 


2.78 


190 


*** 


2.17 


2108 


Retn to next year 


71% 


194 


*** 


55% 


2302 



Valid n (excluding missing values) upon which indicator is based. 

See Appendix 1 for details on the statistical tests used to assess group differences. 
•p<.05; •*p<.01; 
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Table 10. Peer Mentoring Comparisons Among First-Time 
Students Who Placed into Remedial Reading 



3dbkgfbund 'andjEn^ilmentiiCh^ ■ . 




Participants 




Non-Partic. 




Indicator 


n’ 


sig.‘ 


Indicator 




Average Age 


19.7 


46 




19.4 


596 


Percent Female 


44% 


46 




55% 


596 


Percent Minority 


17% 


46 




25% 


596 


Avg. HS % Class Rank 


45.2 


42 


* 


42.8 


516 


Avg Pet Need Met 


48.3 


33 




38.1 


374 


Placed into Rem Math 


87% 


46 




84% 


595 


Placed into Rem Writing 


67% 


46 




72% 


583 


Pet in Prep Program 


94% 


46 




79% 


596 


Avg. Semester Hours 


12.0 


46 


** 


10.8 


596 


Perlormance and Persistence r Unadjusted 




Avg Semester GPA 


2.66 


46 


*** 


1.85 


563 


Avg. Mentor Course GPA 


2.65 


45 


*** 


1.85 


522 


Retn to next year 


70% 


46 


*** 


53% 


596 



'Valid n (excluding missing vaiues) upon which indicator is based. 

^See Appendix 1 for details on the statisticaJ tests used to assess group differences. 
•p<.05;**p<.01;***p<.001 



Since so many students participate in the Student 
Mentoring program, it was also possible to look at 
difference in responses to items on the Continuing Student 
Satisfaction and Priorities Survey conducted in Spring 
1996. The survey sample included 174 students who 
participated in the student mentoring program in either or 
both the Fall 1995 and Spring 1996 semester as well as 
391 from the non-participant groups. Among the 57 
satisfaction items, there were statistically significant group 
differences for only two items, but these were telling 
differences. Table 1 1 sununarizes the few attitudinal 
items from among the 100 such items on the survey, for 
which there were statistically significant differences 
between the program participants and non-participants. 
Although one might expect there to be a handful of 



significant differences among so many items just by 
chance, it is clear from the nature of these items, that the 
program has an impact on more than just students grades 
and likelihood of persisting. 

The student satisfaction survey also provides a chance to 
examine other possible differences in the demographics of 
participants and non-participants. For example, while a 
slightly lower percentage of program participants work for 
pay while attending school (71%, compared to 80% of 
non-participants), those who do work average roughly the 
same number of hours per week. Slightly larger 
proportions of program participants are married (32% 
compared to 23%), live with a spouse or partner (43% 
compared to 32%), and have children at home (34%, 
compared to 27%). There was no difference between the 



Table 11. Differences Between Mentoring Program Participants and Non-Participants 
in Student Satisfaction Survey Responses 



Participants 


Non-Participants 


sig. 


Item 


Mean 


N 


Mean 


N 


Social experiences' 


0.41 


166 


0.14 


368 


** 


• >1 
1 uiuriny services 


0.84 


155 


0.38 


266 


*** 


Worked/Studied with students outside class^ 


2.19 


172 


1.66 


385 


*** 


Have class w/community service component^ 


0.51 


172 


0.34 


385 


* 


1 am often bored in class^ 


-0.53 


166 


-0.23 


381 


** 



^Responses provided on a 5-point scale where: -2 = very dissatisfied; -1 = dissatisfied; 0 = neutral; 

-f 1 =; satisfied, and -f2 =: very satisfied 

^Responses provided on a 4-point scale where: 0 = never; 1 = one or two times; 2 = three or four times; and 
3 = five or more times 

^Responses provided on a 5-point scale where: -2 = strongly disagree; -1 = disagree; 0 = neutral; -*>1 s agree; 
and -f2 s strongly agree 

•p<.05; **p<.01 ; •**p<.001 , all tests based on one-way analysis of variance (F-Test). 
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groups in the percent who represent first-generation 
college students from among their families (54% of both 
groups). 

Conclusions 

The Student Mentoring Program has a significant impact 
on students who participate, both in terms of enhanced 
grades and higher retention rates. When looking at the 
entire student population for whom this program is 
available, it appears that many of the relatively better 
prepared students take advantage of this opportunity. 
However, even when controlling for differences in 
students’ level of preparation, there remains a large effect 
of one-third of a full letter grade on average, and between 
a 12 percent higher one-year retention rate. Furthermore, 
the program appears to have even greater benefit for first- 
time students, including the least well prepared among 
them as indicated by placement into remedial reading. 
These analyses do not consider the motivational factors 
that may account for why certain students choose to 
participate and others do not. But it is clear that this 
program offers benefits to students who are at different 
points in their undergraduate career. It also appears that 
students who participate in this program feel better about 
the social aspects of their college experience as a result of 
working with other students outside class. 

Discussion 

The academic support programs considered in this study 
provide a range of services to a diverse array of students. 

It is clear that some of these programs have a significant 
impact on students on average while for others the average 
impact is less clear. 

It is important to note several limitations of this or any 
evaluative study that examines average group differences, 
based on a limited set of input and outcome measures. 
While every attempt was made to identify the most 
important baseline demographic and enrollment 
characteristics^ our efforts are clearly limited to the kinds 
of data available from student records. Furthermore, 
changes in group averages may gloss over other types of 
changes in group characteristics, such as amount of 
variation and distribution shape. 

The present study was also limited by time spans in two 
important ways. First, many of these programs are 
relatively new and program administrators are still 
discovering which procedures and practices work well and 
which do not. Second, the “follow-up” period for 
assessing performance and persistence is relatively short. 
The impact of some of these programs on students, 
especially those geared toward high-risk students, may 
require more time to take effect. 

Since this study assesses impact with regard to but a 
limited set of student outcomes, it may not capture more 



important factors of student development. It also does not 
address at all the impact of these programs on other 
individuals, such as students’ families, employers, and 
campus faculty and staff. Perhaps the best example of this 
last limitation is in relation to the Learning Communities 
program. This program’s development and administration 
has placed a large number of senior faculty members in 
closer contact with undergraduate students, counselors, 
librarians, and technology support staff. The kinds of 
benefits that may accrue from these relationships are far 
beyond the ability of a few simple student outcome 
measures to capture. Furthermore, these benefits may 
take time to develop their full impact. It will be 
interesting to monitor changes in perceptions of student 
welfare through the existing campus surveys of students, 
faculty and staff as these support programs develop 
further. 

These programs vary a great deal in their organization and 
scope. The ARCHE program is primarily a means for^- 
tracking students, to provide for ongoing analyses of the 
students most at risk at lUPUI. The MAP project “ups the 
ante” to some degree by linking involvement with student 
financial aid. The Student Financial Aid Office joined in 
this partnership with the goal of providing certain funds 
tied to a lowered course load for students at risk with 
increased services. This was based upon the assumption 
that academic achievement and persistence are unduly 
impacted in a negative way by students registering for too 
many hours and subsequently not having the time to 
become involved enough with their educational pursuits. 
Despite the offer of NO indebtedness, only three of the 88 
students invited into the program in Fall 1996 completed 
the semester. Students made the choice to register for 
more hours and, in most cases, to increase their level of 
indebtedness. It will be important for the campus to 
investigate other means of limiting course enrollments or 
to link academic experience with work. 

The Horizons program was developed as a 
“compensating” mechanism when the campus changed its 
retention guidelines. Students who would have been 
dismissed were permitted to continue to enroll, with the 
workshop and a counseling appointment deemed a 
minimal “intervention” with these students. It is perhaps 
surprising that a two-hour workshop would have any 
impact on students. Clearly, further research is needed. 

The student mentoring programs, here seen to have the 
most significant and lasting impact, are centered on 
student learning. The mentoring sessions are designed to 
build upon the course material, to provide students with 
the means to become more involved with their learning 
and to begin to learn in collaborative contexts. It could be 
argued that the program’s effectiveness is a function of its 
connection with the classroom. lUPUI students come to 
campus to go to class. And, with this program recording 
some 1,500 student visits a week, students are making use 
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of this resource. It should be noted that the mentoring 
program has a rich body of data that will be helpful as 
qualitative research is developed, primarily consisting of 
journal entries from student mentors. From the beginning 
of the program, students in mentor roles have been asked 
to keep journals as a means of reflecting upon and 
learning from their involvement. Given the increasing 
attention to reflection as a part of the learning process, 
these data will be especially meaningful. 

The Learning Communities effort, centering on the 
Joining the Scholarly Community course and its various 
iterations, is the most extensive program of academic 
support. It provides an instructional team (faculty, 
academic advisor, and peer mentor attending all sessions, 
joined by a librarian and technical support person in 
planning the course and in offering content to some course 
meetings). It is intentionally linked to the student 
mentoring program in that a key role of peer mentor is to 
introduce students to mentoring sessions in other classes. 
Faculty, staff, and students involved with this program are 
conducting extensive research on its effectiveness, both in 
the short-term and in the long-run. 

It is also important to note that the primary contact of 
students with the campus is in the classroom. Program 
reviews and other assessment measures have been 
developed at the departmental and school levels to assess 
the effectiveness of these curricular efforts. Such data are 
particularly important in mathematics, writing, and 
reading, the foundational areas of study for lUPUI’s 
entering students. As the campus continues its efforts to 
serve entering students and to increase student academic 
achievement and persistence, assessment, including the 
issuing of research briefs on a regular basis, will play an 
increasingly important role. Both quantitative and 
qualitative research efforts will be developed, building 
upon the work reported here. 

Finally, it should be noted that the purpose of this study 
was, in effect, three-fold: to evaluate the short-term impact 
of the programs being incorporated into University 
College; to establish a baseline of these effects as a way to 
monitor program effectiveness in the future; and to 
promote the use of systematic evaluation in formulating 
decisions about program development and support. 
Ongoing efforts in the Undergraduate Education Center to 
track studeni participdiion in ihese programs made this 
study possible. It is important to maintain such efforts so 
that we may be more deliberate in our efforts to assess the 
effectiveness of all campus programs. 
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Appendix 1. Variables used in Comparing Program and Comparison Groups 



Indicator 


Statistical Test for Assessing Group Differences 
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Background and Enrollment Characteristics 
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Average Age 


One-way ANOVA (F-Test) on age to nearest year 
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Percent Female 


Chi-square test for independence with gender (male/female) 


X 


X 


X 


X 


X 


Percent African American 


Chi-square test for independence with ethnic status as African 
American/ail others 
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Percent Minority 


Chi-square test for independence with ethnic status as minoritv/all others 
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Average High School Class 
Rank 


One-way ANOVA (F-Test) on percentile rank in high school class 
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Apply for Financial Aid 


Chi-square test for independence with categories, application on file/no 
application on file 
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Average Percent of Need 
Met 


One-way ANOVA (F-Test) on percent of financial aid need that was met 
among those who had a financial aid application 






X 


X 


X 


Percent in Prep Program 


Chi-square test for independence with categories UEC-P, UEC, or All 
Others. 
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Percent Beginner 


Chi-square test for independence with categories Beginner, Transfer 
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Percent Freshmen 


Chi-square test for Independence with all class level categories 
(freshman, sophomore, junior, senior) 
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Placed into Remedial Math 
(MOIOor 001) 


Chi-square test for independence of four placement categories: M010, 
001 , 1 1 1 , or College Level 
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Placed into Remedial 
Writing (EOlOor WOOD 


Chi-square test for independence of three placement categories: E010, 
wool, W 1 31. Placement into ESL excluded 
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Placed into Remedial 
Reading (Gateway, X150, 
or X152) 


Chi-square test for independence of four placement categories: 
Gateway, X150, XI 52, or College Level. 
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Average Semester Hours 


One-way ANOVA (F-Test) on student course credit load as of census 
date. 
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Average Prior GPA 


One-way ANOVA (F-Test) on grade-point average prior to “treatment" 
semester 
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Performance and Persistence Indicators 












Average Semester GPA 


One-way ANOVA (F-Test) on grade-point average for the “treatmenr 
semester 
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Grades in Courses: 
ENG wool 


One-way ANOVA on average grade-points (e.g., A=4, B=3. etc.) among 
students who completed the course. 
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ENG W131 
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EDUC X150 
EDUC X151 
EDUC X152 
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MATH M010 
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MATH 001 
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MATH 1 1 1 
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PSY B104 
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Percent Completing 
Mentoring Course 


Chi-square test for independence on categories of 100%, 75%, 50%, 
and 25%, reflecting the percent of mentoring courses completed by each 
student who took up to four such courses in a semester. 
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Retention to Next Semester 


Chi-square test for Independence on students who returned versus did 
not return for the subsequent semester 
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Retention to Next Year 


Chl-square test for independence on students who returned versus did 
not return for two semesters later (Fall to Fall or Spring to Sprina) 
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